Issues in the determination of “responders” and “non-responders” in physiological research

As a follow-up to our 2015 review, we cover more issues on the topic of “response heterogeneity”, which we define as clinically-important individual differences in the physiological responses to the same treatment or intervention that cannot be attributed to random within-subjects variability. We highlight various pitfalls with the common practice of counting the number of “responders”, “non-responders” and “adverse responders” in samples that have been given certain treatments/interventions for research purposes. We focus on the classical parallel-group randomised controlled trial (RCT) design and assume typical good practice in trial design.We show that sample responder counts are biased because individuals differ in terms of pre-to-post within-subjects random variability in the study outcome(s) and not necessarily treatment response. Ironically, sample differences in responder counts may be explained wholly by sample differences in mean response, even if there is no response heterogeneity at all. Sample comparisons of responder counts also have relatively low statistical precision. These problems do not depend on how the response threshold has been selected, e.g. on the basis of a measurement error statistic, and are not rectified fully by the use of confidence intervals for individual responses in the sample